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SUMMARY 

This report describes objective measurements of signal-to-noise ratios and 
of waveform distortion using a 'pulse— and-bar* test waveform. The results of these 
measurements are compared with subjective assessments. The agreement is fairly good 
in view of the fact that the objective measurements did not include noise or waveform 
distortion occurring at the picture source, although the subjective assessments would 
have been influenced by any such noise or distortion. 

The results show that if the peak-to-peak noise is less than 8$ of the 
double— amplitude-peak video signal, as measured with a waveform monitor, the most 
probable subjective assessment of overall picture quality would be "good". If the 
noise-to— signal ratio is between 856 and 20$, an assessment of "fairly good" is 
indicated. 

As regards waveform distortion, the results show that a distortion rating of 
3$ would indicate a 90$ probability of a subjective assessment of overall quality in 
the grade 1 or 2, "excellent or good". A rating of 7"5$ indicates a 50$ probability 
of such an assessment. 

1. INTRODUCTION 

The subjective assessments made during the comparison tests carried out under 
the auspices of the T. A,C./T»S, C. and B.B.C, Working Party in 1957/8 have been fully 
described elsewhere. 1 These tests extended over several months and necessitated 
large numbers of measurements being made in a wide range of reception conditions. 
Specially equipped mobile laboratories travelled along radial routes from Crystal 
Palace making assessments of reception conditions at sites along the routes. It was 
decided to take advantage of the opportunity, thus provided, to make some objective 
measurements at each site for comparison with the subjective assessments. The 
measurements were not completely objective because they required visual assessment of 
trace displacements on a cathode— ray tube screen. However, as this assessment was 
aided by the use of superimposed graticules and masks, it was felt that such measure- 
ments would be considerably more independent of the observer than was the case with 
purely subjective assessments of picture quality judged by visual inspection of a 
picture monitor. Besides measurements of the ratio of signal— to— noise or —inter- 
ference, it was decided to make measurements of waveform distortion using the pulse— 
and-bar test waveform. 2 Although these pulse- and-bar tests have proved extremely 
useful in measuring the performance of television links, they were originally intended 



for use with noise-free, linear networks, but it was hoped that the usefulness of the 
method for assessing the service area of a vestigial— sideband television broadcasting 
station might be assessed. The main purpose of the programme was to make the 
subjective assessments and the limited amount of time available for the objective 
measurements made it impossible to make more than a few simple measurements at each site. 

This report describes the measurements made by the B.B.C. mobile laboratory 
and the degree of agreement between the objective measurements and the subjective 
assessments. 

2. THE OBJECTIVE MEASUREMENTS 

The mobile laboratory 3 was equipped with apparatus for the simultaneous 
reception of a television signal having a carrier frequency of 45 Mc/s (Channel 1) and 
one having a carrier frequency of 654*25 Mc/s (Band "V) . The video waveforms obtained 
from these signals were supplied to a picture monitor for subjective assessment and 
to a precision waveform monitor for objective measurements. One of these measurements 
consisted of visual estimation of the peak-to-peak deflexion of the cathode-ray tube 
trace due to the video waveform and that due to the random noise, or interference. 
By using one line of each picture (during the field blanking period) for the noise 
measurement, the error introduced by mains hum was reduced. This also ensured that 
the noise was not due to the picture source but was a function of the site and field 
strength. 

The same monitor was used for measuring the waveform distortion. For this 
purpose a special pulse— and-bar test waveform 2 was transmitted. The test waveform 
was inserted in the field blanking period (line number 11) thus causing no inter- 
ference with the picture, and was viewed on the monitor through an edge-illuminated 
perspex mask engraved with limit patterns which enabled the various symptoms of wave- 
form distortion to be evaluated. The pulse was approximately "sine squared" in shape 
with a half- amplitude duration of 0°33/i-s*, the bar waveform had a duration of 
approximately 40fis, and both it and the pulse had amplitudes which were 70$ of the 
excursion from blanking level to peak white. These measurements, which were carried 
out only during the Series A (405-line) tests, corresponded to a simplified version of 
the "routine test method" described by Lewis. 2 The 0*33/U.s pulse and the 40 /Ms bar 
were Lewis's "2T pulse" and "bar". 

Lewis's method entailed measurements of the following characteristics." 

a) The response to a 50 c/s square-wave test signal, 

;b) The uniformity of the 40 /xs bar waveform, 

to) The relative amplitudes of the 2T-pulse and bar waveforms. 

;d) The width at half amplitude of the 2T-pulse. 

(e) The response to the 2T— pulse, weighted according to the time difference 
relative to the centre of the pulse, but ignoring the central region 
between the half-amplitude points. This measurement included the "skirts" 
of the pulse, any oscillations before or after it, and any echoes. 

;f) Various features of the response to a 0'17/J.a T-pulse. 

Of these characteristics, only (b), (c) and (e) were measured. By means of the masks, 
numerical values could be assigned to them and the biggest of the four ratings 
(referred to later as "maximum rating <f,") was taken for comparison with the subjective 



assessment of picture quality. This procedure did not take full advantage of the 
test waveform for diagnosis or measurement of distortion, but was adopted in view of 
the limited time available for the measurements and the subsequent computation. 
Nevertheless, if the measurements could be shown to correlate well with the subjective 
assessments, a simple and useful procedure would be confirmed for future use. 



3. THE SUBJECTIVE ASSESSMENTS 

These assessments were made separately by two observers seated 4 ft (1*2 m) 
from a 21 in, (53 em! picture monitor. The lighting in the mobile laboratory was 
subdued but sufficient for the observers to read and fill in their test report forms. 
The observers made assessments of seventeen parameters of the picture. We are here 
concerned only with two of these parameters^ they are;' 

I. Noise (random) which was assessed using the scale? 

1 Imperceptible 

2 Just perceptible 

3 Definitely perceptible but not disturbing 

4 Somewhat objectionable 

5 Definitely objectionable 

6 Unusable 

II, Overall assessment of picture using the scales 

1 Excellent 

2 Good 

3 Fairly good 

4 Rather poor 

5 Poor 

6 Very poor 



4. RESULTS 

4olo General 

It was found that the method of objective measurement used was very unsuit- 
able for measuring ignition interference because only one line in each picture was 
inspected. This, and the fact that the objective and subjective measurements were 
not made at exactly the same instant, resulted in a poor correlation between the sets 
of measurements for ignition and similar erratic interference, 

When continuous wave interference was encountered it was usually accompanied 
by a high level of random noise. Small amounts of c.w. interference are often 
difficult to observe in the presence of a high level of random noise, and it was found 
that often the objective measurement recorded c.w. interference when the subjective 
assessment did not include reference to this defect, and vice versa. All sites at 
which either measurement or assessment Included c.w, interference have been omitted 
from this analysis. Similarly, if ignition Interference was predominant in either 
test, the site has been omitted from this analysis. 



4.2. Noise 

The statistical frequency distribution of subjective assessment of noise for 
various objective noi se— to— signal ratios is tabulated in Table 1 for Series A, 
Channel 1, and in Table 2 for Series A, Band V. The same information for Series B 
results is shown in Tables 3 and 4. These tables show the mean subjective noise 
grade, and the standard deviation from that mean, for each noise— to— signal ratio. 



TABLE 1 
CHANNEL 1, SERIES A 
Distribution of noise measurements against noise assessments 



Peak-to-peak noise ^ 
d. a, p. video signal 


No. of 
Samples 


Per cent samples in 
subjective noise assess- 
ment grade heading column 


Mean 

subjective 

grade 


Standard 
deviation 
from mean 


1 


2 


3 


4 


5 


6 


2-5 


4 


50 


25 


25 








1-75 


0-85 


4'0 


20 


30 


55 


15 








1-85 


0*65 


5-0 


31 


23 


71 


6 








1-85 


0-5 


6-0 


28 


7 


57 


32 


4 






2-3 


0-65 


7-5 


22 


27 


46 


27 








2-0 


0*75 


10-0 


33 




52 


45 


3 






2-5 


0-55 


12-5 


12 




50 


50 








2-5 


0-5 


20-0 


12 






42 


50 


8 




3*9 


0-65 



TABLE 2 

BAND V, SERIES A 

Distribution of noise measurements against noise assessments 







Per cent samples in 


Mean 


Standard 


Peak— to— peak noise „, 
i 


No. of 


subjective noise assess- 


subjective 


deviation 


d. a. p. video signal 


samples 


ment grade heading column 


grade 


from mean 


1 


2 


3 


4 


5 


6 


2*5 


34 


15 


70 


15 








2*0 


0-55 


3-5 


42 


12 


78 


10 








2*0 


0-45 


4-0 


110 


15 


67 


18 








2-0. 


0-45 


5-0 


119 


9 


56 


35 








2-3 


0-6 


6*0 


46 


9 


54 


35 


2 






2-3 


0-6 


7-5 


51 


6 


20 


57 


17 






2-6 


0*8 


10-0 


63 




19 


60 


18 


3 




3-0 


0-7 


15-0 


56 




5 


55 


36 


4 




3-4 


0-65 


20-0 


21 






29 


66 


5 




3-6 


0-5 


25-0 


10 






20 


60 


20 




4-0 


0*65 


30-0 


20 








45 


55 




4-5 


0-5 


40-0 


36 








45 


47 


8 


4-6 


0-65 


50-0 


20 






10 


25 


55 


10 


4-65 


0-8 


60*0 


14 








7 


72 


21 


5'1 


0-5 


75-0 


10 










60 


40 


5*4 


0-5 


100- + 


28 










21 


79 


5*8 


0*4 



TABLE 3 
CHANNEL 1, SERIES B 
Distribution of noise measurements against noise assessments 



Peak-to-peak noise . 

f, 

d. a. p . vi deo s ign al 


No. of 
samples 


Per cent samples in 
subjective noise assess- 
ment grade heading column 


Mean 

subjective 

grade 


Standard 
deviation 
from mean 


1 


2 


3 


4 


5 


6 


2-0 


4 


75 


25 










1-25 


0-45 


4*0 


18 


67 


33 










1-3 


0-45 


5-0 


22 


68 


32 










1-3 


0-45 


6*0 


12 


17 


83 










1'8 


0-4 


7*5 


10 




90 


10 








2'1 


0*3 


10-0 


16 


6 


82 


12 








2*1 


0-45 


15*0 


8 






75 


25 






3-3 


0-45 


25*0 


8 






50 


38 


12 




3-6 


0-7 


40-0 


6 






50 


50 






3*5 


0-7 


50-0 


8 








25 


75 




4-8 


0*45 


75-0 


4 










75 


25 


5-25 


0-45 


100-0 + 


5 








20 


40 


40 


5*2 


0-75 



Distribution of noise 



TABLE 4 

BAND V, SERIES B 
measurements against noise assessments 



Peak-to-peak noise ^ 


No. of 
samples 


Per cent samples in 
subjective noise assess- 
ment grade heading column 


Mean 

subjective 

grade 


Standard 
deviation 
from mean 


d. a. p. video signal 


1 


2 


3 


4 


5 


6 


1-0 


8 


88 


12 










1-1 


0-35 


1-5 


8 


100 












1-0 


o-o 


2-0 


20 


80 


20 










1-2 


0*4 


2-5 


8 


88 


12 










1-1 


0-35 


3'0 


18 


94 


6 










1-05 


0-25 


4*0 


32 


72 


28 










1-3 


0-45 


5-0 


56 


61 


37 


2 








1-4 


0-55 


6-0 


44 


36 


64 










1-6 


0-5 


7-5 


32 


34 


63 


3 








1-7 


0-5 


9-0 


15 


20 


73 


7 








1-95 


0-45 


10' 


42 


5 


64 


31 








2-3 


0-55 


12-5 


24 


4 


50 


46 








2-4 


0-6 


15-0 


44 




39 


50 


11 






2-7 


0-6 


17-5 


24 




17 


79 


4 






2-85 


0-45 


20'0 


20 




5 


80 


15 






3-1 


0-45 


22-5 


23 




9 


43 


48 






3-4 


0-65 


25-0 


26 






34 


58 


8 




3-75 


0-6 


30*0 


12 






33 


25 


42 




4-1 


0-85 


35*0 


12 






33 


50 


17 




3-8 


0-65 


40-0 


18 






28 


44 


28 




4-05 


0-75 


50-0 


29 








38 


62 




4*65 


0-5 


60-0 


26 








11 


62 


27 


5*15 


0-6 


75-0 


22 








9 


64 


27 


5-2 


0-55 


100-0 + 


56 








2 


21 


77 


5*75 


0-5 



The standard deviation is less than one grade in all cases and is frequently below a 
half— grade. The mean values from the tables are plotted in Pig. 1. The curves for 
Series A have higher subjective grades than those for Series B for any given noise-to- 
signal ratio. The Series A tests were carried out in the winter of 1957/8 and the 
Series B tests in the following spring and summer and it is possible that some change 
in subjective criterion may have occurred over this time. It is thought, however, that 
one important cause of the discrepancy was due to the fact that, for Series A, the 
test pictures consisted of normal programmes, whilst in Series B only films and slides 
were used. The objective measurement ignored any noise originating in the programme 
source and transmitted with the picture, but the subjective assessment of noise would, 
of course, include this. The Series A curves have an asymptote of about 1*8 on the 
subjective— grade scale. The Series B curves show this grade to be equivalent to 
noise with a peak-to— peak amplitude equal to 8$ of the d. a. p. video signal, and this 
is a very reasonable average for normal programmes, including winter outdoor broadcasts, 
etc. Prom this it appears that the average noise grading of pictures transmitted in the 
winter of 1957/8 was about 2 or "just perceptible". Similarly it would appear that 
the average noise grading of the pictures transmitted in the Series B tests, films and 
slides, was about 1*2 or almost "imperceptible". In spite of these differences 
between the Series A and Series B results, the agreement is fairly good and the method 
of measuring noise-to-signal ratio with a waveform monitor gives close agreement with 
subjective noise assessments. It is interesting to see that the Series B curves for 
Channel 1 and Band V agree so closely, in view of the fact that the former used 
405— line standards while the latter used the 625— line standards adopted by Western 
European countries. This result appears to conflict with the results of video 
comparisons carried out by Weaver. 4 
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Fig. I - Mean subjective noise assessment plotted against noise level 

^— Series A Channel I 

Series A Band T 

Series B Channel 1 

Series B Band T 



The Series A results were also used in order to compare the objective noise- 
to-signal ratios with the subjective assessment of overall picture quality. Tables 5 
and 6 show the statistical frequency distributions, mean overall subjective grading 
and standard deviations from the means for various noise-to-signal ratios, Table 5 
showing the results for those sites having subjective noise gradings of 1, 2 or 3, and 
Table 6 the results for sites having subjective noise gradings of 4, 5 or 6. The 
reason for dividing the results in this fashion was that it was thought that if noise 
were sufficiently bad to be graded 4, 5 or 6, this would affect the overall assessment 
of picture quality and a reasonably close correlation would exist. On the other hand, 
if the noise were subjectively in grades 1, 2 or 3, a less close relationship between 
noise and overall assessment might result because other defects such as waveform 
distortions might predominate. The expected result did not appear, as can be seen by 
comparing the standard deviations in Table 5 with those in Table 6. The spread of 
results is scarcely greater for the low— noise sites than for the high-noise sites. 
The mean subjective grade for overall assessment is plotted against peak-to-peak 
noise/d. a. p. video signal in Pig. 2. Separate curves were derived from Table 5 and 
Table 6, and a third curve was obtained from all the observations. Again, it is of 
interest to note that the curve has an asymptote of 2 for the minimum overall subjective 
grading, as was the case for subjective noise gradings in Series A. Prom this curve 
it appears that if the peak— to-peak noise is less than 8$ of the d. a. p. signal, 
measured as described in Section 2, the most probable subjective overall assessment 
will be grade 2 or "good". Between 8$ and 20$ of noise would indicate a probable 
assessment of grade 3, "fairly good", and a noise level between 20$ and 40$ should 
produce an assessment in grade 4, "rather poor". 
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Fig. 2 - Mean overall subjective assessment plotted against noise level 
(Channel 1, Series A, and Band X, Series A) 



All samples 



B Only samples with subjective noise 

assessment in Grades I, 2 or 3. 

C Only samples with subjective noise 

assessment in Grades 4, 5 or 6. 



TABLE 5 



SERIES A, CHANNEL 1 AND BAND V 



Distribution of noise measurements against overall assessment 
for samples having noise assessments 1, 2 or 3 



Peak— to— peak noise 
d. a. p. video signal " 


No. of 
samples 


Per cent samples in 
overall subjective assess- 
ment grade heading column 


Mean 

subjective 

grade 


Standard 
deviation 
from mean 


1 


2 


3 


4 


5 


6 


2-5 


32 


19 


69 


12 








1-95 


0-55 


3*0 


30 


20 


67 


10 


3 






1-9 


0-65 


3-5 


16 


19 


69 


12 








1-95 


0'55 


4-0 


135 


12 


68 


17 


3 






2*1 


0-6 


5-0 


153 


5 


67 


22 


4 


2 




2-3 


0'75 


6-0 


67 


12 


55 


31 


2 






2-2 


0-65 


7*5 


54 


2 


63 


35 








2*3 


0-5 


9-0 


9 




34 


66 








2-7 


0*5 


10*0 


73 




36 


44 


18 


2 




2-9 


0-8 


12*5 


26 




11 


62 


23 


4 




3*2 


0-7 


15*0 


17 




35 


41 


24 






2-9 


0*75 


17-5 


11 




18 


73 


9 






2'9 


0-5 


20-0 


9 




22 


78 








2-8 

... 


0*45 



TABLE 6 

SERIES A, CHANNEL 1 AND BAND V 



Distribution of noise measurements against overall assessment 
for samples having noise assessments 4, 5 or 6 



Peak-to-peak noise 
d. a. p. video signal * 


No. of 
samples 


Per cent samples in 
overall subjective assess- 
ment grade heading column 


Mean 

subjective 

grade 


Standard 
deviation 
from mean 


1 


2 


3 


4 


5 


6 


5-0 


8 




12 


38 


25 


25 




3-6 


0'95 


7*5 


10 






20 


80 






3-8 


0-45 


10-0 


16 






44 


44 


6 


6 


3-7 


0-8 


15*0 


25 




4 


16 


68 


12 




3-7 


0-7 


20-0 


20 






40 


50 


10 




3-7 


0-65 


25*0 


13 






23 


23 


54 




4*2 


0-85 


30-0 


23 






22 


35 


43 




4-2 


0-8 


35-0 


15 






20 


40 


40 




4-2 


0-75 


40-0 


20 






10 


45 


20 


25 


4'6 


1-0 


50-0 


9 






22 


22 


56 




4*3 


0-8 


60-0 


15 










66 


34 


5*3 


0-6 


75*0 


10 










34 


66 


5'7 


0-6 


100-0 + 


26 










19 


81 


5-8 


0-45 



4.3. Waveform Distortion 

The maximiim rating for waveform distortion at each site was compared with 
the subjective grading for overall picture quality. For this comparison it was 
decided to omit all sites where the presence of c.w. interference was reported by the 
subjective observer and all sites reported unusable because of random noise. Tables 
7 and 8 show the statistical frequency distributions of sites in the various subjective 
grades, tabulated against the maximum rating, together with the mean subjective grades 
and the standard deviations from those means. Two extra columns are added at the 
right of the tables, which show the probable percentage of sites which would occur 
within ±0*5 grade and + 1'5 grade of the mean, assuming that the distribution follows 
the 'normal error' curve. Table 7 shows the results for Series A, Channel 1, and 
Table 8 shows those for Series A, Band V. The mean values from Tables 7 and 8 are 
plotted respectively in Pigs. 3(a) and 3(b). 

The results show that the waveform distortion measurement is of limited 
value in these conditions. Ratings better than 3% or 4£ do not result in better 
assessments because the transmitted picture quality (including the effect of camera 
noise, etc.) is not sufficiently good for the mean subjective grading to improve 
beyond grade 2. The curves are restricted at the other end because the number of 
sites in which the rating exceeded 10$ was statistically negligible. 

The test waveform was inserted into the signal at the Switching Centre, 
Broadcasting House, which has a very good link to Crystal Palace. Thus, the objective 
measurement of waveform distortion did not include any distortion due to the programme 
source (at Lime Grove) or links to Broadcasting House. The maximum rating measured 
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Fig. 3 - Mean subjective overall assessment plotted against 
maximum waveform distortion rating 



(a) Channel 1, Series A 

(b) Band Y, Series A 
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TABLE 7 



CHANNEL 1, SERIES A 



Distribution of waveform distortion measurements 
against overall picture quality assessments 







Per cent samples in over- 




Standard 


Probable 


per cent 


Maximum 
rating 


No. of 
samples 


all subjective assessment 
grade heading column 


Mean 
assessment 


deviation 
from mean 


within + x of mean 


1 


2 


3 


4 


5 


6 


x = 0*5 


x = 1-5 


1-0 


4 




100 










2-0 


o-o 






2-0 


172 


IS 


75 


8 


1 






2-0 


0'5 


68 


99 


2-5 


45 


18 


71 


11 








2-0 


0-5 


68 


99 


3-0 


125 


14 


74 


12 








2*0 


0-5 


68 


99 


4-0 


129 


14 


74 


12 








2*0 


0-5 


68 


99 


5-0 


73 


3 


72 


18 


4 


3 




2*3 


0-75 


50 


95 


6'0 


19 


5 


48 


42 


5 






2-5 


0*7 


53 


97 


7-0 


15 




74 


13 


13 






2*4 


0-7 


53 


97 


9*0 


17 




36 


18 


34 


12 




3-2 


1-05 


37 


84 


35-0 


2 






50 


50 






3-5 


o-o 







TABLE 8 



BAND V, SERIES A 



Distribution of waveform distortion measurements 
against overall picture quality assessments 







Per cent samples in over- 




Standard 


Probable 


per cent 


Maximum 
rating 


No. of 
samples 


all subjective assessment 
grade heading column 


Mean 
assessment 


deviation 
from mean 
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by the mobile laboratory at a test site on Farthing Downs in Surrey was repeatedly 
confirmed as being between If, and 2%. Therefore, in general, ratings greater than 2$, 
were due to the propagation path between Crystal Palace and the site being tested. 
It appears that this rating can be as bad as 4# before the mean assessment of quality 
is appreciably deteriorated. 

Pig. 4 shows a graph of the percentage of assessments which are in grades 
1 and 2, "excellent" and "good", against the maximum rating factor. It will be seen 
that for a rating of approximately 7'E#, half the assessments will bein grades 1 and 8 
and the other half in grade 3 or worse. Prom this it would appear that a useful quick 
check on the probable subjective assessment of a given site can be obtained by 
measuring the distortion rating as described. If the maximum figure is less than 
7*5# there is a 50$ probability that the assessment will be "good" or "excellent". 
Furthermore, if the maximum rating is less than 3% there is a 00% probability of the 
assessment being "good" or "excellent". 
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Fig 4 - Proportion of subjective assessments in Grades I and 2 
plotted against maximum rating for waveform distortion 
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5. CONCLUSIONS 

The semi-objective method of measuring noise-to-signal ratio agrees very 
closely with subjective assessments of noise level and also of overall picture quality. 
This merely confirms the well known fact that, statistically, the most important single 
parameter determining the goodness of a receiving site is the received field strength. 
For a level of peak-to-peak noise less than 8<t of the double- amplitude-peak video 
signal, measured as described in Section 2, the most probable subjective overall 
assessment is "good"; noise levels between 856 and Z0% should produce assessments of 
"fairly good". Noise levels in excess of 20% should result in assessments of "rather 
poor", "poor" or "very poor". 

The waveform distortion tests can be used to predict the probable subjective 
assessment of picture quality as affected by waveform distortion produced by the 
propagation path. A maximum ratio of 3% will have a 90$ probability of being 
assessed "good" or "excellent", and a rating of 7" 5$ will have a 50$ chance of being 
so assessed. 

It must be remembered, however, that c. w. and impulsive interference have 
not been taken into account in arriving at these conclusions. 
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